Control device, control method, and computer-readable recording medium

ABSTRACT

A control device includes circuitry configured to: obtain an external environment recognition image representing a recognition result of a surrounding area of a moving body; obtain obstacle information representing a detection result of an obstacle in the surrounding area of the moving body; and display the obstacle information on a display device in a superimposed manner on the external environment recognition image. The obstacle information indicates a region in which the obstacle is detected among a plurality of fan-shaped detection regions centered at the moving body when viewed from above the moving body, and a size of a central angle of the fan-shaped detection region is set according to a distance from the moving body.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 fromJapanese Patent Application No. 2022-039039 filed on Mar. 14, 2022, thecontents of which are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to a control device, a control method,and a computer-readable recording medium.

BACKGROUND

In recent years, efforts have been actively made to provide access tosustainable transportation systems in consideration of vulnerabletraffic participants. In order to achieve this goal, attention isfocused on research and development to further improve safety andconvenience of traffic through research and development related to adriving assistance technique.

The related art discloses a technique of displaying an image of asurrounding area of a vehicle captured by an in-vehicle camera or thelike on a display device of the vehicle during low-speed movement suchas parking. A known technique displays information on an obstacledetected by a sensor or the like in a superimposed manner on an image ofa surrounding area of a vehicle when displaying the image of thesurrounding area of the vehicle. Japanese Patent Application Laid-OpenPublication No. 2020-182220 (hereinafter, referred to as PatentLiterature 1) discloses a top view image generation device that detectsan obstacle by a sensor or the like and changes a display position of anicon indicating the obstacle with respect to an icon indicating a hostvehicle according to a distance to the detected obstacle when displayinga top view image of a surrounding area of the vehicle.

Japanese Patent Application Laid-Open Publication No. 2006-341641(hereinafter, referred to as Patent Literature 2) discloses an imagedisplay device that displays, when an obstacle is detected by anobstacle detection unit, obstacle information indicating the detectedobstacle in a direction of the detected obstacle in a superimposedmanner on a direct image captured by a camera and a top view image of asurrounding area of a vehicle created by using the direct image.

Patent Literature 1 discloses that the display position of the obstacleinformation indicating the obstacle is changed according to the distancefrom the vehicle to the obstacle, but there is no description regardingrespective appearance of the obstacle information at different displaypositions. In addition, with the image display device according toPatent Literature 2, it is possible to display the top view image inwhich the obstacle information indicating the obstacle is reproduced ina distorted form, and the direct image in which the obstacle informationis expressed in a form that is less distorted and closer to the actualobstacle. However, there is no description regarding improvement ofappearance of the obstacle information in the top view image. Therefore,in a driving assistance technique, there is room for improvement in away of displaying obstacle information on an obstacle that is displayedin a superimposed manner on a peripheral image of a surrounding area ofa vehicle.

The present disclosure provides a control device, a control method, anda computer-readable recording medium storing a control program thatenable to display information on an obstacle detected in an image of asurrounding area of a vehicle in a superimposed manner in a form thatdoes not cause any sense of discomfort. This contributes to developmentof a sustainable transportation system.

SUMMARY

A first aspect of the present disclosure relates to a control deviceincluding circuitry configured to:

obtain an external environment recognition image representing arecognition result of a surrounding area of a moving body;

obtain obstacle information representing a detection result of anobstacle in the surrounding area of the moving body; and

display the obstacle information on a display device in a superimposedmanner on the external environment recognition image, in which

the obstacle information indicates a region in which the obstacle isdetected among a plurality of fan-shaped detection regions centered atthe moving body when viewed from above the moving body, and

a size of a central angle of the fan-shaped detection region is setaccording to a distance from the moving body.

A second aspect of the present disclosure relates to a control methodperformed by a control device, including:

obtaining an external environment recognition image representing arecognition result of a surrounding area of a moving body,

obtaining obstacle information representing a detection result of anobstacle in the surrounding area of the moving body, and

displaying the obstacle information on a display device in asuperimposed manner on the external environment recognition image, inwhich

the obstacle information indicates a region in which the obstacle isdetected among a plurality of fan-shaped detection regions centered atthe moving body when viewed from above the moving body, and

a size of a central angle of the fan-shaped detection region is setaccording to a distance from the moving body.

A third aspect of the present disclosure relates to a non-transitorycomputer-readable recording medium storing a control program for causinga processor of a control device to execute a process, the processincluding:

obtaining an external environment recognition image representing arecognition result of a surrounding area of a moving body,

obtaining obstacle information representing a detection result of anobstacle in the surrounding area of the moving body, and

displaying the obstacle information on a display device in asuperimposed manner on the external environment recognition image, inwhich

the obstacle information indicates a region in which the obstacle isdetected among a plurality of fan-shaped detection regions centered atthe moving body when viewed from above the moving body, and

a size of a central angle of the fan-shaped detection region is setaccording to a distance from the moving body.

According to the present disclosure, it is possible to provide a controldevice, a control method, and a computer-readable recording mediumstoring a control program capable of displaying information on anobstacle detected in an image of a surrounding area of a vehicle in asuperimposed manner in a form that does not cause any sense ofdiscomfort.

BRIEF DESCRIPTION OF DRAWINGS

Exemplary embodiment(s) of the present invention will be described indetail based on the following figures, wherein:

FIG. 1 is a side view showing an example of a vehicle whose movement iscontrolled by a control device according to an embodiment;

FIG. 2 is a top view of the vehicle shown in FIG. 1 ;

FIG. 3 is a block diagram showing an internal configuration of thevehicle shown in FIG. 1 ;

FIG. 4 is a flow chart showing display control performed by acalculation unit on an external environment recognition image andobstacle information;

FIG. 5 shows an example of a detection region that can be detected by anobstacle detection unit:

FIG. 6 shows an example of obstacle information displayed on a frontrecognition image; and

FIG. 7 shows an example of obstacle information displayed on a top viewimage.

DESCRIPTION OF EMBODIMENTS

Hereinafter, an embodiment of a control device according to the presentdisclosure will be described with reference to the accompanyingdrawings. The drawings are viewed in directions of reference signs. Inaddition, in the present specification and the like, in order tosimplify and clarify the description, a front-rear direction, aleft-right direction, and an up-down direction are described accordingto directions viewed from a driver of a vehicle 10 shown in FIGS. 1 and2 . In the drawings, a front side of the vehicle 10 is denoted by Fr, arear side thereof is denoted by Rr, a left side thereof is denoted by L,a right side thereof is denoted by R, an upper side thereof is denotedby U, and a lower side thereof is denoted by D.

<Vehicle 10 Whose Movement is Controlled by Control Device According toPresent Disclosure>

FIG. 1 is a side view of the vehicle 10 whose movement is controlled bythe control device according to the disclosure. FIG. 2 is a top view ofthe vehicle 10 shown in FIG. 1 . The vehicle 10 is an example of amoving body in the disclosure.

The vehicle 10 is an automobile including a drive source (not shown) andwheels including driving wheels driven by power of the drive source andsteering wheels that are steerable. In the present embodiment, thevehicle 10 is a four-wheeled automobile including a pair of left andright front wheels and a pair of left and right rear wheels. The drivesource of the vehicle 10 is, for example, an electric motor. The drivesource of the vehicle 10 may also be an internal combustion engine suchas a gasoline engine or a diesel engine, or a combination of an electricmotor and an internal combustion engine. In addition, the drive sourceof the vehicle 10 may drive the pair of left and right front wheels, thepair of left and right rear wheels, or four wheels, that is, the pair ofleft and right front wheels and the pair of left and right rear wheels.The front wheels and the rear wheels may both be steering wheels thatare steerable, or one of the front wheels and the rear wheels may besteering wheels that are steerable.

The vehicle 10 further includes side mirrors 11L and 11R. The sidemirrors 11L and 11R are mirrors (rearview mirrors) that are providedoutside front seat doors of the vehicle 10 for the driver to check arear side and a rear lateral side. Each of the side mirrors 11L and 11Ris fixed to a body of the vehicle 10 by a rotation shaft extending in avertical direction and can be opened and closed by rotating about therotation shaft.

The vehicle 10 further includes a front camera 12Fr, a rear camera 12Rr,a left side camera 12L, and a right side camera 12R. The front camera12Fr is a digital camera that is provided at a front portion of thevehicle 10 and captures an image of a front side of the vehicle 10. Therear camera 12Rr is a digital camera that is provided at a rear portionof the vehicle 10 and captures an image of a rear side of the vehicle10. The left side camera 12L is a digital camera that is provided on theleft side mirror 11L of the vehicle 10 and captures an image of a leftside of the vehicle 10. The right side camera 12R is a digital camerathat is provided on the right side mirror 11R of the vehicle 10 andcaptures an image of a right side of the vehicle 10.

<Internal Configuration of Vehicle 10>

FIG. 3 is a block diagram showing an example of an internalconfiguration of the vehicle 10 shown in FIG. 1 . As shown in FIG. 3 ,the vehicle 10 includes a sensor group 16, a navigation device 18, acontrol electronic control unit (ECU) 20, an electric power steering(EPS) system 22, and a communication unit 24. The vehicle 10 furtherincludes a driving force control system 26 and a braking force controlsystem 28.

The sensor group 16 obtains various detection values used for controlperformed by the control ECU 20. The sensor group 16 includes the frontcamera 12Fr, the rear camera 12Rr, the left side camera 12L, and theright side camera 12R. In addition, the sensor group 16 includes a frontsonar group 32 a, a rear sonar group 32 b, a left side sonar group 32 c,and a right side sonar group 32 d. In addition, the sensor group 16includes wheel sensors 34 a and 34 b, a vehicle speed sensor 36, and anoperation detection unit 38. The sensor group 16 may include a radar.

The front camera 12Fr, the rear camera 12Rr, the left side camera 12L,and the right side camera 12R obtain recognition data (for example, anexternal environment recognition image) for recognizing a surroundingarea of the vehicle 10 by capturing images of the surrounding area ofthe vehicle 10. An external environment recognition image captured bythe front camera 12Fr is referred to as a front recognition image. Anexternal environment recognition image captured by the rear camera 12Rris referred to as a rear recognition image. An external environmentrecognition image captured by the left side camera 12L is referred to asa left side recognition image. An external environment recognition imagecaptured by the right side camera 12R is referred to as a right siderecognition image. An image formed by the left side recognition imageand the right side recognition image may be referred to as a siderecognition image. An external environment recognition image generatedby synthesizing imaging data of the front camera 12Fr, the rear camera12Rr, the left side camera 12L, and the right side camera 12R may bereferred to as a top view image of the vehicle 10.

The front sonar group 32 a, the rear sonar group 32 b, the left sidesonar group 32 c, and the right side sonar group 32 d emit sound wavesto the surrounding area of the vehicle 10 and receive reflected soundsfrom other objects. The front sonar group 32 a includes, for example,four sonars. The sonars constituting the front sonar group 32 a arerespectively provided on an obliquely left front side, a front leftside, a front right side, and an obliquely right front side of thevehicle 10. The rear sonar group 32 b includes, for example, foursonars. The sonars constituting the rear sonar group 32 b arerespectively provided on an obliquely left rear side, a rear left side,a rear right side, and an obliquely right rear side of the vehicle 10.The left side sonar group 32 c includes, for example, two sonars. Thesonars constituting the left side sonar group 32 c are provided in thefront of a left side portion of the vehicle 10 and the rear of the leftside portion, respectively. The right side sonar group 32 d includes,for example, two sonars. The sonars constituting the right side sonargroup 32 d are provided in the front of a right side portion of thevehicle 10 and the rear of the right side portion, respectively. Thefront sonar group 32 a, the rear sonar group 32 b, the left side sonargroup 32 c, and the right side sonar group 32 d obtain detection data(for example, obstacle information) for detecting an obstacle in thesurrounding area of the vehicle 10.

The wheel sensors 34 a and 34 b detect a rotation angle of the wheel ofthe vehicle 10. The wheel sensors 34 a and 34 b may be implemented byangle sensors or displacement sensors. The wheel sensors 34 a and 34 boutput detection pulses each time the wheel rotates by a predeterminedangle. Detection pulses outputted from the wheel sensors 34 a and 34 bare used to calculate the rotation angle of the wheel and a rotationspeed of the wheel. A movement distance of the vehicle 10 is calculatedbased on the rotation angle of the wheel. The wheel sensor 34 a detects,for example, a rotation angle θa of the left rear wheel. The wheelsensor 34 b detects, for example, a rotation angle θb of the right rearwheel.

The vehicle speed sensor 36 detects a speed of a vehicle body of thevehicle 10, that is, a vehicle speed V. and outputs the detected vehiclespeed V to the control ECU 20. The vehicle speed sensor 36 detects thevehicle speed V based on, for example, rotation of a countershaft of atransmission.

The operation detection unit 38 detects a content of an operationperformed by a user using an operation input unit 14 and outputs thedetected content of the operation to the control ECU 20. The operationinput unit 14 includes, for example, various user interfaces such as aside mirror switch that switches opened and closed states of the sidemirrors 11L and 11R, and a shift lever (a select lever or a selector).

The navigation device 18 detects a current position of the vehicle 10 byusing, for example, a global positioning system (GPS), and guides theuser along a route toward a destination. The navigation device 18includes a storage device (not shown) that includes a map informationdatabase.

The navigation device 18 includes a touch panel 42 and a speaker 44. Thetouch panel 42 functions as an input device and a display device of thecontrol ECU 20. The speaker 44 outputs various types of guidanceinformation to the user of the vehicle 10 by voice.

The touch panel 42 is configured to input various commands to thecontrol ECU 20. For example, the user can input a command via the touchpanel 42 to display the external environment recognition image of thevehicle 10. In addition, the touch panel 42 is configured to displayvarious screens related to a control content of the control ECU 20. Forexample, the external environment recognition image of the vehicle 10 isdisplayed on the touch panel 42. Constituent elements other than thetouch panel 42, for example, a head-up display (HUD), a smartphone, or atablet terminal may be used as the input device or the display device.

The control ECU 20 includes an input and output unit 50, a calculationunit 52, and a storage unit 54. The calculation unit 52 is implementedby, for example, a central processing unit (CPU). The calculation unit52 performs various types of control by controlling each unit based on aprogram stored in the storage unit 54. In addition, the calculation unit52 receives and outputs signals from and to each unit connected to thecontrol ECU 20 via the input and output unit 50.

The calculation unit 52 includes an external environment recognitionunit 55 that recognizes the external environment recognition image, adisplay control unit 56 that controls display of the externalenvironment recognition image, and an obstacle detection unit 57 thatrecognizes an obstacle in the surrounding area of the vehicle 10. Thecalculation unit 52 is an example of a control device in the disclosure.

The external environment recognition unit 55 obtains, from each camera,the external environment recognition image representing a recognitionresult of a peripheral image of the vehicle 10 captured by the frontcamera 12Fr, the rear camera 12Rr, the left side camera 12L, and theright side camera 12R.

The obstacle detection unit 57 obtains, from each sonar, obstacleinformation representing a detection result of an obstacle in thesurrounding area of the vehicle 10 detected by the front sonar group 32a, the rear sonar group 32 b, the left side sonar group 32 c, and theright side sonar group 32 d. For example, the obstacle detection unit 57obtains, from each sonar, obstacle information representing a detectionresult of an obstacle detected in the surrounding area of the vehicle 10when the vehicle 10 is parked or exits a parking space. Examples of theobstacle include an object that can interfere with traveling of thevehicle 10, such as a person, an animal, another vehicle, and aplanting. The parking or exit of the vehicle 10 may be parking or exitimplemented by autonomous driving, driving assistance, or manualdriving. In addition, the detection device that detects the obstacle inthe surrounding area of the vehicle 10 may be, for example, a LIDAR, aradar, or the cameras 12Fr, 12Rr, 12L, and 12R in addition to thesonars.

The display control unit 56 displays the external environmentrecognition image obtained by the external environment recognition unit55 on a display device of the vehicle 10. Specifically, the displaycontrol unit 56 displays, on the touch panel 42, the front recognitionimage captured by the front camera 12Fr, the rear recognition imagecaptured by the rear camera 12Rr, the left side recognition imagecaptured by the left side camera 12L, and the right side recognitionimage captured by the right side camera 12R. In addition, the displaycontrol unit 56 displays, on the touch panel 42, the top view image ofthe vehicle 10 generated by synthesizing the imaging data of the frontcamera 12Fr, the rear camera 12Rr, the left side camera 12L, and theright side camera 12R.

In addition, the display control unit 56 displays the obstacleinformation on the touch panel 42 in a superimposed manner on theexternal environment recognition image. For example, the display controlunit 56 displays the obstacle information on the touch panel 42 in asuperimposed manner on the front recognition image, the rear recognitionimage, the left side recognition image, the right side recognitionimage, and the top view image.

Specifically, when a predetermined input is performed by the user of thevehicle 10, the display control unit 56 displays the obstacleinformation on the touch panel 42 in a superimposed manner on theexternal environment recognition image. Examples of the predeterminedinput performed by the user include an operation input from the touchpanel 42 and an operation of turning on a switch. In addition, when amoving state of the vehicle 10 is a predetermined state, the displaycontrol unit 56 displays the obstacle information on the touch panel 42in a superimposed manner on the external environment recognition image.Examples of the predetermined moving state include a case where amovement speed of the vehicle 10 is equal to or less than apredetermined speed (for example, 15 km/h), a case where it isrecognized based on GPS information, the external environmentrecognition image, or the like that the vehicle 10 moves into apredetermined parking lot region, and a case where a gear of the vehicle10 is shifted to a predetermined position.

The EPS system 22 includes a steering angle sensor 100, a torque sensor102, an EPS motor 104, a resolver 106, and an EPS ECU 108. The steeringangle sensor 100 detects a steering angle θst of a steering wheel 110.The torque sensor 102 detects a torque TQ applied to the steering wheel110.

The EPS motor 104 applies a driving force or a reaction force to asteering column 112 connected to the steering wheel 110, therebyenabling assistance of an operation performed by an occupant on thesteering wheel 110 and enabling autonomous steering during parkingassistance. The resolver 106 detects a rotation angle θm of the EPSmotor 104. The EPS ECU 108 controls the entire EPS system 22. The EPSECU 108 includes an input and output unit (not shown), a calculationunit (not shown), and a storage unit (not shown).

The communication unit 24 enables wireless communication with anothercommunication device 120. The other communication device 120 is a basestation, a communication device of another vehicle, a smartphone or atablet terminal carried by the user of the vehicle 10, or the like.

The driving force control system 26 includes a driving ECU 130. Thedriving force control system 26 executes driving force control of thevehicle 10. The driving ECU 130 controls a driving force of the vehicle10 by controlling an engine or the like (not shown) based on anoperation performed on an accelerator pedal (not shown) by the user.

The braking force control system 28 includes a braking ECU 132. Thebraking force control system 28 executes braking force control of thevehicle 10. The braking ECU 132 controls a braking force of the vehicle10 by controlling a brake mechanism (not shown) or the like based on anoperation performed on a brake pedal (not shown) by the user.

<Example of Control Performed by Calculation Unit 52>

Next, an example of display control performed by the calculation unit 52of the vehicle 10 on the external environment recognition image and theobstacle information will be described with reference to a flow chartshown in FIG. 4 .

For example, the user who drives the vehicle 10 attempts to park thevehicle 10 in a parking space in a parking lot. For example, it isassumed that the vehicle 10 is traveling at a low speed (for example, 15km/h or less) or is stopped. For example, when the traveling state ofthe vehicle 10 is detected based on a detection value of the vehiclespeed sensor 36, the calculation unit 52 of the vehicle 10 startsprocessing shown in FIG. 4 .

The display control unit 56 of the calculation unit 52 determineswhether to display the external environment recognition image obtainedby the external environment recognition unit 55 on the touch panel 42 ofthe navigation device 18 (step S11). The external environmentrecognition image is displayed, for example, when a parking button (notshown) displayed on the touch panel 42 is tapped. Therefore, forexample, the display control unit 56 determines whether the parkingbutton is tapped by the user.

In step S11, when the external environment recognition image is not tobe displayed on the touch panel 42, that is, when the parking button isnot tapped (step S11: No), the display control unit 56 repeats theprocessing of step S11 and stands by until the parking button is tapped.

In step S11, when the external environment recognition image is to bedisplayed on the touch panel 42, that is, when the parking button istapped (step S11: Yes), the display control unit 56 displays theexternal environment recognition image on the touch panel 42 (step S12).For example, when the gear of the vehicle 10 is set to drive, neutral,or parking, the display control unit 56 displays the front recognitionimage captured by the front camera 12Fr on the touch panel 42. Inaddition, for example, when the gear of the vehicle 10 is set toreverse, the display control unit 56 displays the rear recognition imagecaptured by the rear camera 12Rr on the touch panel 42. In addition,when the top view image is selected by the user, the display controlunit 56 displays the top view image on the touch panel 42.

Next, the obstacle detection unit 57 of the calculation unit 52determines whether an obstacle is detected in the surrounding area ofthe vehicle 10 (step S13). As described above, the obstacle in thesurrounding area of the vehicle 10 can be detected based on the obstacleinformation obtained by the sonar groups 32 a to 32 d. Examples of theobstacle information include information on a direction of the obstaclewith respect to the vehicle 10, information on a distance from thevehicle 10 to the obstacle, and information on in which preset detectionregions the obstacle is detected. The preset detection regions will bedescribed later with reference to FIG. 5 .

In step S13, when no obstacle is detected in the surrounding area of thevehicle 10 (step S13: No), the obstacle detection unit 57 repeats theprocessing of step S13 and stands by until an obstacle is detected.

In step S13, when an obstacle is detected in the surrounding area of thevehicle 10 (step S13: Yes), the obstacle detection unit 57 outputsobstacle information on the detected obstacle to the display controlunit 56. The display control unit 56 displays the obstacle informationoutputted from the obstacle detection unit 57 on the touch panel 42 in asuperimposed manner on the external environment recognition image (stepS14). As described in step S12, the external environment recognitionimage includes the front recognition image, the rear recognition image,the top view image, and the like. The obstacle information displayed ina superimposed manner on the external environment recognition image willbe described later with reference to FIGS. 6 and 7 .

Next, the display control unit 56 determines whether to hide theexternal environment recognition image displayed on the touch panel 42(step S15). For example, when a parking completion button (not shown)displayed on the touch panel 42 is tapped, the external environmentrecognition image becomes hidden. Therefore, for example, the displaycontrol unit 56 determines whether the parking completion button istapped by the user.

In step S15, when the external environment recognition image is hidden,that is, when the parking completion button is not tapped (step S15:No), the display control unit 56 returns to step S13 and repeats eachprocessing.

In step S15, when the external environment recognition image is hidden,that is, when the parking completion button is tapped (step S15: Yes),the display control unit 56 hides the external environment recognitionimage displayed on the touch panel 42 (step S16), and ends the presentprocessing.

<Detection Region and Obstacle Information>

Next, the detection region for detecting the obstacle and the obstacleinformation indicating the detection result of the obstacle will bedescribed with reference to FIG. 5 .

FIG. 5 shows a detection region 60 formed in the surrounding area of thevehicle 10 as viewed from above the vehicle 10. As shown in FIG. 5 , thedetection region 60 is a region formed around the detection device (forexample, the sonar groups 32 a to 32 d) mounted on the vehicle 10 and isformed as a region in which an obstacle can be detected. For example,the detection region 60 includes a front detection region 61 formedaround the front sonar group 32 a, a rear detection region 62 formedaround the rear sonar group 32 b, a left side detection region 63 formedaround the left side sonar group 32 c, and a right side detection region64 formed around the right side sonar group 32 d.

The front detection region 61 is a detection region formed by aplurality of fan shapes centered at the front sonar group 32 a. The reardetection region 62 is a detection region formed by a plurality of fanshapes centered at the rear sonar group 32 b. Each fan-shaped detectionregion is a detection region in a range in a traveling direction of thevehicle 10, that is, a range on the front side Fr or the rear side Rr ofthe vehicle 10. In addition, the left side detection region 63 is adetection region formed by a plurality of rectangles centered on theleft side sonar group 32 c. The right side detection region 64 is adetection region formed by a plurality of rectangles centered on theright side sonar group 32 d. Each rectangular detection region is adetection region in a range in a vehicle width direction of the vehicle10, that is, a range on the left side L or the right side R of thevehicle 10.

The front sonar group 32 a and the rear sonar group 32 b can detect anobstacle within a range of a detection distance LV in a fan-shapeddetection range in the traveling direction of the vehicle 10. The leftsonar group 32 c and the right sonar group 32 d can detect an obstaclewithin a range of a detection distance LH in a rectangular detectionrange in the vehicle width direction of the vehicle 10. The detectiondistance LV in the traveling direction is configured to be longer thanthe detection distance LH in the vehicle width direction.

The front detection region 61 formed by the plurality of fan shapesincludes an inner detection region group 61 a disposed on a side closeto the vehicle 10 and an outer detection region group 61 b disposed on aside farther from the vehicle 10 than the inner detection region group61 a. The inner detection region group 61 a is an example of a firstdetection region group in the present disclosure. The outer detectionregion group 61 b is an example of a second detection region group inthe present disclosure. Since the front detection region 61 and the reardetection region 62 have the same configuration, the followingdescription will focus on the front detection region 61.

A size of a central angle of each of the plurality of fan-shapeddetection regions is set according to a distance from the vehicle 10 tothe detection region. Specifically, the size of the central angle of thefan shape in a detection region disposed on a side far from the vehicle10 is smaller than the size of the central angle of the fan shape in adetection region disposed on a side close to the vehicle 10. In theexample shown in the drawing, the inner detection region group 61 a isformed by one fan-shaped inner detection region portion 61 al, whereasthe outer detection region group 61 b is divided into two fan shapes,that is, outer detection region portions 61 b 1 and 61 b 2. Accordingly,a central angle θ2 of the outer detection region portions 61 b 1 and 61b 2 is smaller than a central angle θ1 of the inner detection regionportion 61 al. In addition, the number of outer detection regionportions is larger than the number of inner detection region portions.

In the detection region configured as described above, the sonar groupof the vehicle 10 detects in which fan shape or in which rectangle anobstacle in the surrounding area of the vehicle 10 is located. The sonargroups 32 a to 32 d detect a direction and a distance of the obstaclebased on sound waves emitted to the surrounding area of the vehicle 10and reflected sounds received from the obstacle, and detect a positionof the fan shape or rectangle where the obstacle is present based on thedetected direction and distance. A position of the detected obstacle isdisplayed in the region where the obstacle is detected as the obstacleinformation indicating the detection result of the obstacle. Forexample, when the obstacle is detected in the front detection region 61or the rear detection region 62, the obstacle information is representedas corresponding to the size of the central angle of the fan shape ofthe region in which the obstacle is detected.

Specifically, in FIG. 5 , an obstacle 71 on a right front side of thevehicle 10 is an obstacle detected in the fan shape of the outerdetection region portion 61 b 2 in the front detection region 61.Therefore, obstacle information on the obstacle 71 is displayed as, forexample, band-shaped obstacle information 81 extending corresponding tothe central angle θ2 of the fan shape of the outer detection regionportion 61 b 2. In addition, an obstacle 72 on a right rear side of thevehicle 10 is an obstacle detected in the fan shape of the innerdetection region portion 61 al in the rear detection region 62.Therefore, obstacle information on the obstacle 72 is displayed as, forexample, band-shaped obstacle information 82 extending corresponding tothe central angle θ1 of the fan shape of the inner detection regionportion 61 al.

<Obstacle Information Displayed on External Environment RecognitionImage>

Next, an example of the obstacle information displayed in a superimposedmanner on the external environment recognition image will be describedwith reference to FIGS. 6 and 7 .

FIG. 6 shows an example of the front recognition image captured by thefront camera 12Fr of the vehicle 10 and obstacle information displayedin a superimposed manner on the front recognition image. For example, itis assumed that the vehicle 10 in FIG. 6 is searching for a parkingspace in which the vehicle 10 is to be parked in a certain parking lot.

As shown in FIG. 6 , a front recognition image 91 captured by the frontcamera 12Fr is displayed on the touch panel 42. In the front recognitionimage 91, an image of a person (obstacle) 73 is captured on the rightfront side of the vehicle 10. The front sonar group 32 a detects adirection and a distance of the person 73 with respect to the vehicle 10and detects a position of a fan shape where the person 73 is present. Atthe detected position of the fan shape, obstacle informationcorresponding to a central angle (see FIG. 5 ) of the fan shape isdisplayed as band-shaped obstacle information 83 in a superimposedmanner at the position of the person 73 in the front recognition image91. When the user of the vehicle 10 notices the person 73 and stops bystepping on the brake, a message 92 b such as “please directly check thesurrounding area” is displayed on the front recognition image 91 on thetouch panel 42.

FIG. 7 shows an example of the top view image generated based on imagingdata of the cameras 12Fr, 12Rr, 12L, and 12R including the frontrecognition image shown in FIG. 6 and obstacle information displayed ina superimposed manner on the top view image. Similarly to FIG. 6 , thevehicle 10 in FIG. 7 is searching for a parking space in which thevehicle 10 is to be parked in a certain parking lot.

As shown in FIG. 7 , the touch panel 42 displays a top view image 92generated based on the imaging data of the cameras 12Fr, 12Rr, 12L, and12R. In the top view image 92, the image of the person (obstacle) 73 iscaptured on the right front side of the vehicle 10, as in FIG. 6 . Inaddition, the position of the fan shape where the person 73 is presentis detected based on the direction and the distance of the person 73with respect to the vehicle 10, which are detected by the front sonargroup 32 a, and the band-shaped obstacle information 83 corresponding tothe central angle (see FIG. 5 ) of the fan shape at the detectedposition of the fan shape is displayed in a superimposed manner at theposition of the person 73. The top view image 92 may be displayed on thetouch panel 42 side by side with the front recognition image 91 shown inFIG. 6 .

In a configuration of the detection region formed in the surroundingarea of the vehicle 10 in the related art, for example, in the detectionregion 60 shown in FIG. 5 , the central angle θ1 of the inner detectionregion portion 61 al in the inner detection region group 61 a of thefront detection region 61 and the rear detection region 62 is the sameas the central angle θ2 of the outer detection region portion 61 b 1 inthe outer detection region group 61 b (θ1=θ2). That is, the outerdetection region portion in the outer detection region group 61 b isformed as one outer detection region portion. Therefore, when theobstacle 72 in FIG. 5 is detected, the same band-shaped obstacleinformation 81 as the obstacle information 82 shown in FIG. 5 isdisplayed in a superimposed manner, whereas band-shaped obstacleinformation extending corresponding to the central angle θ1=θ2, forexample, band-shaped obstacle information 181 indicated by a broken lineis displayed in a superimposed manner when the obstacle 71 is detected.Therefore, band-shaped obstacle information superimposed on a fartherobstacle is longer in an arc direction, which is excessively large withrespect to the obstacle displayed in an actual camera image, and thusthere is a sense of discomfort.

On the other hand, in a case where the obstacle information 83indicating the detection result of the person (obstacle) 73 is displayedin a superimposed manner on the external environment recognition image(the front recognition image 91 and the top view image 92), the displaycontrol unit 56 of the calculation unit 52 in the present embodimentdisplays the obstacle information 81 and 82 in the regions (the innerdetection region portion 61 a 1 and the outer detection region portion61 b 2) in which the obstacles 71 and 72 are detected among theplurality of fan-shaped detection regions 60 centered on the vehicle 10when viewed from above the vehicle 10, for example, as shown in FIG. 5 .The size of the central angle of the fan-shaped detection region 60 isset according to the distance from the vehicle 10. With thisconfiguration, the obstacle information 81 and 82 can be displayed in anaccurate range regardless of the distance between the vehicle 10 and theobstacles 71 and 72. Accordingly, a sense of discomfort that theobstacle information is displayed excessively large with respect to theobstacle displayed as an actual camera image can be prevented, and thussafety of the vehicle 10 can be improved.

In addition, the front detection region 61 and the rear detection region62 formed by the plurality of fan shapes include the inner detectionregion group 61 a disposed on the side close to the vehicle 10 and theouter detection region group 61 b disposed on the side farther from thevehicle 10 than the inner detection region group 61 a. The central angleθ2 of the outer detection region portions 61 b 1 and 61 b 2 in the outerdetection region group 61 b is smaller than the central angle θ1 of theinner detection region portion 61 al in the inner detection region group61 a Accordingly, since the outer detection region portions 61 b 1 and61 b 2 of the outer detection region group 61 b that is far from thevehicle 10 are arranged more finely than the inner detection regionportion 61 al of the inner detection region group 61 a that is close tothe vehicle 10, even when an obstacle is present in the distance, anobstacle range indicated by the obstacle information can be preventedfrom being excessively wide relative to an actual size of the obstacle,and thus it is possible to improve the safety of the vehicle 10.

In addition, the detection distance LV within which the front sonargroup 32 a and the rear sonar group 32 b can detect an obstacle in thetraveling direction of the vehicle 10 is longer than the detectiondistance LH within which the left side sonar group 32 c and the rightside sonar group 32 d can detect an obstacle in the vehicle widthdirection of the vehicle 10. Each fan-shaped detection region is adetection region in a range in the traveling direction of the vehicle10, that is, a range on the front side Fr or the rear side Rr of thevehicle 10. Accordingly, by setting the detection distance LV in therange in the traveling direction of the vehicle 10 to be longer, adistant obstacle can be detected at an early stage. Therefore, it ispossible to reduce a risk of contact with the obstacle due to movementof the vehicle 10, and thus the safety of the vehicle 10 can beimproved.

In addition, for example, when the user of the vehicle 10 performs anoperation input to the touch panel 42, the display control unit 56displays the obstacle information on the touch panel 42 in asuperimposed manner on the external environment recognition image.Accordingly, it is possible to display the obstacle information in asuperimposed manner on the external environment recognition image whenit is necessary to display the obstacle information, and thus the safetyof the vehicle 10 is improved.

In addition, when the movement speed of the vehicle 10 is equal to orless than a predetermined speed or when it is recognized that thevehicle 10 moves into a predetermined parking lot region, the displaycontrol unit 56 displays the obstacle information on the touch panel 42in a superimposed manner on the external environment recognition image.Accordingly, it is possible to display the obstacle information in asuperimposed manner on the external environment recognition image whenit is necessary to display the obstacle information, and thus the safetyof the moving body is improved.

Although the embodiment of the present disclosure is described above,the present invention is not limited to the above embodiment, andmodifications, improvements, and the like can be made as appropriate.

For example, although the case where the vehicle 10 is parked isdescribed in the above embodiment, the same control can be performedwhen the vehicle 10 exits.

The control method described in the above embodiment can be implementedby executing a control program prepared in advance on a computer. Thecontrol program is recorded in a computer-readable storage medium and isexecuted by being read from the storage medium. In addition, the controlprogram may be provided in a form stored in a non-transitory storagemedium such as a flash memory or may be provided via a network such asthe Internet. The computer that executes the control program may beprovided in a control device, may be provided in an electronic devicesuch as a smartphone, a tablet terminal, or a personal computer capableof communicating with the control device or may be provided in a serverdevice capable of communicating with the control device and theelectronic device.

In addition, at least the following matters are described in the presentspecification. Although corresponding constituent elements and the likein the above embodiment are shown in parentheses, the present inventionis not limited thereto.

(1) A control device including:

an external environment recognition unit (external environmentrecognition unit 55) configured to obtain an external environmentrecognition image representing a recognition result of a surroundingarea of a moving body (vehicle 10);

an obstacle detector (obstacle detection unit 57) configured to obtainobstacle information (obstacle information 81, 82) representing adetection result of an obstacle (obstacle 71, 72) in the surroundingarea of the moving body; and

a display controller (display control unit 56) configured to display theobstacle information on a display device (touch panel 42) in asuperimposed manner on the external environment recognition image, inwhich

the obstacle information indicates a region in which the obstacle isdetected among a plurality of fan-shaped detection regions (detectionregion 60) centered at the moving body when viewed from above the movingbody, and

a size of a central angle of the fan-shaped detection region is setaccording to a distance from the moving body.

According to (1), it is possible to display the obstacle information inan accurate range regardless of the distance between the moving body andthe obstacle, and thus safety of the moving body can be improved.

(2) The control device according to (1), in which

the plurality of fan-shaped detection regions include a first detectionregion group and a second detection region group that is farther fromthe moving body than the first detection region group and whose fanshape has a smaller central angle than that of the first detectionregion group.

According to (2), since each detection region of the second detectionregion group that is far from the moving body is arranged more finelythan each detection region of the first detection region group that isclose to the moving body, even when an obstacle is present in thedistance, an obstacle range indicated by the obstacle information can beprevented from being excessively wide with respect to an actual size ofthe obstacle, and thus the safety of the moving body can be improved.

(3) The control device according to (1) or (2), in which

a detection distance of the obstacle detector in a range in a travelingdirection of the moving body is longer than a detection distance of theobstacle detector in a range in a vehicle width direction of the movingbody, and

the plurality of fan-shaped detection regions are provided in a range inthe traveling direction of the moving body.

According to (3), by setting the detection distance in the range in thetraveling direction of the moving body to be longer, a distant obstaclecan be detected at an early stage, and it is possible to reduce a riskof contact with the obstacle due to movement of the moving body, andthus the safety of the moving body can be improved.

(4) The control device according to any one of (1) to (3), in which

the display controller displays the obstacle information on the displaydevice in a superimposed manner on the external environment recognitionimage when a predetermined input is performed by a user of the movingbody.

According to (4), it is possible to display the obstacle information ina superimposed manner in a scene in which it is necessary to display theobstacle information, and thus the safety of the moving body isimproved.

(5) The control device according to any one of (1) to (4), in which

the display controller displays the obstacle information on the displaydevice in a superimposed manner on the external environment recognitionimage when a moving state of the moving body is a predetermined state.

According to (5), it is possible to display the obstacle information ina superimposed manner in a scene in which it is necessary to display theobstacle information, and thus the safety of the moving body isimproved.

(6) A control method performed by a control device, including:

obtaining an external environment recognition image representing arecognition result of a surrounding area of a moving body,

obtaining obstacle information representing a detection result of anobstacle in the surrounding area of the moving body, and

displaying the obstacle information on a display device in asuperimposed manner on the external environment recognition image, inwhich

the obstacle information indicates a region in which the obstacle isdetected among a plurality of fan-shaped detection regions centered atthe moving body when viewed from above the moving body, and

a size of a central angle of the fan-shaped detection region is setaccording to a distance from the moving body.

According to (6), it is possible to display the obstacle information inan accurate range regardless of the distance between the moving body andthe obstacle, and thus safety of the moving body can be improved.

(7) A non-transitory computer-readable recording medium storing acontrol program for causing a processor of a control device to execute aprocess, the process including:

obtaining an external environment recognition image representing arecognition result of a surrounding area of a moving body,

obtaining obstacle information representing a detection result of anobstacle in the surrounding area of the moving body, and

displaying the obstacle information on a display device in asuperimposed manner on the external environment recognition image, inwhich

the obstacle information indicates a region in which the obstacle isdetected among a plurality of fan-shaped detection regions centered atthe moving body when viewed from above the moving body, and

a size of a central angle of the fan-shaped detection region is setaccording to a distance from the moving body.

According to (7), it is possible to display the obstacle information inan accurate range regardless of the distance between the moving body andthe obstacle, and thus safety of the moving body can be improved.

1. A control device comprising circuitry configured to: obtain anexternal environment recognition image representing a recognition resultof a surrounding area of a moving body; obtain obstacle informationrepresenting a detection result of an obstacle in the surrounding areaof the moving body; and display the obstacle information on a displaydevice in a superimposed manner on the external environment recognitionimage, wherein the obstacle information indicates a region in which theobstacle is detected among a plurality of fan-shaped detection regionscentered at the moving body when viewed from above the moving body, anda size of a central angle of the fan-shaped detection region is setaccording to a distance from the moving body.
 2. The control deviceaccording to claim 1, wherein the plurality of fan-shaped detectionregions include a first detection region group and a second detectionregion group that is farther from the moving body than the firstdetection region group and whose fan shape has a smaller central anglethan that of the first detection region group.
 3. The control deviceaccording to claim 1, wherein a detection distance in a range in atraveling direction of the moving body is longer than a detectiondistance in a range in a vehicle width direction of the moving body, andthe plurality of fan-shaped detection regions are provided in a range inthe traveling direction of the moving body.
 4. The control deviceaccording to claim 1, wherein the circuitry displays the obstacleinformation on the display device in a superimposed manner on theexternal environment recognition image when a predetermined input isperformed by a user of the moving body.
 5. The control device accordingto claim 1, wherein the circuitry displays the obstacle information onthe display device in a superimposed manner on the external environmentrecognition image when a moving state of the moving body is apredetermined state.
 6. A control method performed by a control device,comprising: obtaining an external environment recognition imagerepresenting a recognition result of a surrounding area of a movingbody, obtaining obstacle information representing a detection result ofan obstacle in the surrounding area of the moving body, and displayingthe obstacle information on a display device in a superimposed manner onthe external environment recognition image, wherein the obstacleinformation indicates a region in which the obstacle is detected among aplurality of fan-shaped detection regions centered at the moving bodywhen viewed from above the moving body, and a size of a central angle ofthe fan-shaped detection region is set according to a distance from themoving body.
 7. A non-transitory computer-readable recording mediumstoring a control program for causing a processor of a control device toexecute a process, the process comprising: obtaining an externalenvironment recognition image representing a recognition result of asurrounding area of a moving body, obtaining obstacle informationrepresenting a detection result of an obstacle in the surrounding areaof the moving body, and displaying the obstacle information on a displaydevice in a superimposed manner on the external environment recognitionimage, wherein the obstacle information indicates a region in which theobstacle is detected among a plurality of fan-shaped detection regionscentered at the moving body when viewed from above the moving body, anda size of a central angle of the fan-shaped detection region is setaccording to a distance from the moving body.